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ABSTRACT 



This report discusses the outcomes of two studies that 
investigated the effectiveness of the Safe Schools Healthy Students 
Initiative (SS/HSI) , a federal grant program designed to promote healthy 
childhood development and prevent violence and substance abuse. The first 
study, titled "Results of a Longitudinal Study of Disciplinary Referrals in 
an Urban School District," evaluated the success of programs funded by the 
grant in Pinellas County (Florida) schools. It investigated referral patterns 
over a 3-year period from 1998-2000. While results show that violence does 
occur in these schools, the frequency of violence appears to be much lower 
than perceived by the general population. Of the 109,628 students enrolled 
during the 1999-2000 school year, 42,615 students had at least one 
disciplinary referral with at total of 180,912 referrals reported in the 
district. A small percentage of these referrals were either violent or policy 
type referrals. The majority of referrals (80 percent) were in the classroom 
behavior and campus/school rules categories. The number of referrals did not 
change over the 3-year period; however, fighting did decrease. In the second 
study titled "A Factor Analysis of Perceptions of Violence in Pinellas County 
Schools," 101 school counselors were surveyed on school safety and 
familiarity with SS/HSI program strategies. The majority of counselors (60%) 
perceived their school to be safe. Teasing and bullying were their major 
concerns. (CR) 
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Symposium: 

The Safe Schools Healthy Students 
Initiative: The Evaluation Context in 
District Wide Initiatives 



Introduction to the Symposium 

The Safe Schools Healthy Students Initiative is a joint effort 
among the Departments of Juvenile Justice, Education and Health 
and Human Services. Over 70 grants between 1 and 3 million dollars 
have been awarded to address the needs of children in our nations 
schools. The grants were designed to promote healthy childhood 
development and prevent violence and substance abuse. The initiative 
includes a mandate for the evaluation of grant activities, program 
efforts and student outcomes. This symposium concentrates on evaluation methodologies and the 
results of process and contextual analyses of funded sites. The two papers provided here include the 
results of a district wide study identifying staff perceptions of risks and violence in schools, and a 
longitudinal analysis of disciplinary referrals for students in a variety of grant funded programs. These 
presentations provide two methodologies for learning about the context of safety in the schools. 

The survey of school counselors is a straight forward attempt to understand the kinds of problems 
confronting staff in schools. The results serve as a reminder that the extraordinary violence reflected in 
school shootings, while highly visible, may be a less significant threat on a daily basis to students than 
more mundane acts of intimidation, teasing and bullying. We hope that the domains identified in this 
study will serve to classify and explain the experience of violence for our nations school children. 
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document do not necessarily represent 
official OERI position or policy. 
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The second study is a preliminary analysis of disciplinary data maintained by the school district. 
This data set is important both because it serves to illustrate the kinds of behavioral problems 
confronting schools, and because it is a foundation indicator of the health of the school. For a school 
intervention to claim to be successful, the program’s impact must be reflected in the accounts of the 
day-to-day experiences of teachers. 



Results of a Longitudinal Study of Disciplinary Referrals in an 
Urban School District 

Michael Boroughs, Oliver T. Massey & Kathleen Armstrong 

Introduction 

In recent years, violence in the nation’s schools has become a central concern to society. Media 
reports imply that this is a growing problem that must be addressed. One strategy to combat youth 
violence in the schools is intervention via programs to curb violence, reduce substance use and increase 
the perception of safety in the nation’s schools. 

The Safe Schools/Healthy Students Initiative (SS/HSI) is a U.S. government funded grant 
supported by three departments of the government. The Departments of Justice, Health and Human 
Services and Education have collaborated in awarding grant monies to local school districts in an 
effort to fund programs in cooperation with community partners and law enforcement agencies with 
the hope of improving school safety and making students healthier. 

Pinellas County, Florida has approximately one million residents and a large urban school district 
with approximately 1 1 1,000 students. Pinellas is one of four districts in the state of Florida to be 
awarded a grant by the SS/HSI. Part of the Initiative includes a percentage of the funding to include 
an evaluation. The purpose of the evaluation is to measure and report the success or shortcomings of 
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the programs funded by the grant. Our role as evaluators is to collaborate with the district and 
community agencies that provide services in an effort to collect data using multiple methodologies to 
report an accurate reflection of the grant funded programs. 

One source of data includes a referral database provided by the district that is currently available 
for the three years prior to the grant. These data tell us many things about student behaviors during a 
given school semester. This referral database will be maintained throughout the three years of the 
grant. In this database, referrals are grouped into 36 categories. These categories range from minor 
infractions such as “tardiness” to serious or violent acts such as “battery on a student.” Each occurrence 
of these behaviors is collected and organized by individual students referrals, albeit most are included 
in the less severe categories. 

This presentation reported on the analysis of referral patterns over a three-year period in the 
Pinellas County schools from 1998-2000. While the results show that violence does occur in these 
schools, the frequency of violence appears to be much lower than perceived by the general population. 
The benefactors of this information include parents, especially those with kids in the schools; the 
district, particularly administration and professionals who are given an accurate report of what indeed 
takes place; and communities, since everyone lives in some proximity to a school. In addition to 
helping to place media reports into context, this information also helps researchers and evaluators gain 
an understanding of the problems involved, and offers professional techniques to combat them. 

Method 

Participants 

This study was a secondary data analysis. Of the 109,628 students enrolled in the district during 
the 1999-2000 school year, 42,615 students had at least one referral with a total of 180,912 referrals 
reported in the district. The discrepancy between the number of students referred and the number of 
total referrals demonstrates the idea that a small percentage of students generate a high number of 
referrals. There are many repeat offenders with multiple referrals. 

instruments 

Because this was secondary analysis, no instrument was used by the evaluators per se ; instead, 
district-wide referral forms used by school administrators were entered into a spreadsheet by the 
district and the data were sent to the evaluators to be examined and analyzed. These referral forms 
contain 36 behavioral categories along with qualitative notations about the referred students behavior. 
Examples of information on this form include: grade of student, date and time of incident, referring 
teacher, present action, recommendation for further discipline and signature areas for both student 
and parents. 

Analysis 

For the purpose of analysis, five summary categories (called the "Focus 5”) were adopted and 
enhanced from a model already in place in the district. These summary categories are mutually 
exclusive and include all 36 referral types. The Focus 5 category titles are! 1) Policy referrals, which 
require mandatory suspension; 2) Violence; 3) Classroom Behavior; 4) Campus and School Rules; and 
5) Bus Misconduct. This presentation addresses the most pressing issues reflected in the violence and 
policy referral categories, which include possession of substances or weapons. Comparative percentages 
and descriptive statistics are used to highlight the data. 

Results 

Initial results over the three-year period show some interesting findings with relation to the 
perception of increased violence and substance abuse in the schools as reported by media. For 
example, for the 1999 school year, a total of 177,864 referrals were reported with only a small 
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percentage being either violent or policy type referrals. In fact, the classroom behavior and campus/ 
school rules categories account for about 80% of the total referrals in that year and with bus 
misconduct added, these three “less severe” categories account for over 90% of the total referrals. This 
means that less than 10% of referrals included harm to the self or others, carrying a weapon or using 
or possessing an illegal substance. While these results do not suggest that violence and substance abuse 
are negligible, but they do confirm that the perception that violence and policy referrals are the 
majority or even a large minority is misleading. 

The next step was to delve further into the violence and policy referrals to examine exactly what 
had taken place over the three-year period of the baseline study. When highlighting violence and 
policy referrals only, some interesting trends were discovered during the period beginning in the fall of 
1997 and ending in the spring of 2000 (the 1998 through 2000 school years). 

The percentages of total referrals for our two severe categories are as follows. In 1998, policy 
referrals were at 1.4%, in 1999, 1.3% and in 2000, 1.4% of all referrals. Likewise, in 1998 violence 
referrals were 5.5%, in 1999, 5.6% and in 2000, 5.3% of all referrals. In pragmatic terms, there was 
essentially no change over the three-year period. The key finding here is that under 6% of the total 
referrals over three years were violence related while under 2% were policy related, making these two 
severe categories account for just 7% or less of the total number of referrals. 

After each of these categories were deconstructed further, it was found that some shifts have 
occurred. Within the violence category, both battery and sexual harassment remained steady while 
fighting decreased and threats and intimidation increased. Certainly the goal is to eliminate these 
behaviors altogether, but it could be viewed as a positive step, even if temporary, that actually carrying 
out physical violence is reduced in lieu of threats. 

Within the policy category, alcohol and weapons remained constant over the three-year span with 
only negligible increases in these categories. Conversely, a large shift is present here with a great 
reduction in the use of tobacco, which greatly decreased, while the use of other drugs almost doubled. 
Attributions which may account for some of the trends in this category could be: a) the institution of 
zero tolerance policies, or b) drug use increases are clustered particularly around the use of “club 
drugs," such as ecstasy (MDMA) and steroid use in males (Goetz, 2000). 

What is uncategorically the most striking finding in our analysis was the association of school type 
with the percentage of violence referrals. Middle schools had the lowest population of students of all 
non-special schools and yet the highest referral rate for violence. That is, when all violent referrals were 
looked at by school type, we took the 
total number of violent referrals in 
the district over the longitudinal 
period and broke down just these 
violent referrals by the type of school. 

The total population for middle 
schools is less than half of high 
schools and yet the percentage of 
violence referrals has more than 
doubled across all three years (See 
Table 1). 

Discussion 

The information presented in this longitudinal study is tentative because it is baseline data. Only 
after all of the data are submitted for analysis, including the 2003 school year, will we have a clearer 
picture about what, if any, effect programs have on trends in disciplinary referrals. At that point we 
can try to measure changes due to interventions/programs that are funded through the SS/HSI grant. 



Table 1 

Violence Referrals by School Type 





N 


1998 

% 


1999 

% 


2000 

% 


Elementary Schools 


51,380 


20.0 


21.8 


20.8 


Middle Schools 


25.981 


58.6 


57.2 


53.8 


High Schools 


31,116 


13.3 


13.0 


16.0 


Other Schools 


2,459 


8.1 


7.9 


9.3 
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One goal outlined in the grant proposal is for the district to reduce the total number of all referrals 
and also to reduce the severity of the referral types. We have demonstrated, at least initially, that this 
trend has already begun. 

Referrals are a global outcome in that they do not tell us how or why behaviors increase or 
decrease; all trend data tell us is what behaviors changed, if any. Therefore while no concrete 
explanations can be drawn from this analysis, it is helpful in laying the groundwork for beginning the 
evaluation process and prepares us to look at what is to come over the next two years. 

References 

Goetz, J. (2000). Prevalence of substance use among Pinellas County students: The Pinellas drug survey. 
The Juvenile Welfare Board of Pinellas County, Pinellas Park, FL 



A Factor Analysis of Perceptions of Violence in Pinellas 
County Schools 

Joan A. Tucker, Gina Santoro, Oliver T. Massey & Kathleen Armstrong 

Introduction 

Since the early 1990s, rates for both commission of and victimization by homicide in society have 
been declining gradually. However, the profile of offenders of violent crimes has changed dramatically 
(Fox & Zawitz, 2000). The mean age of victims and perpetrators of violent crimes has been decreasing 
since 1976. The rate of homicide victimization in the 14-17 year-old age range increased about 150% 
between 1985 and 1993. In 1998, the rate of homicide victimization in the 18-24 year-old age range 
reflected the highest homicide rate of all age groups. Incidents of homicide committed by teenagers 
14-17 increased also from 1976, reaching a peak in 1993 when the offending rates of 14-17 year-olds 
was higher than the offending rates of 25-49 year-olds. While the incidence of violent crimes has 
decreased overall, younger people often are affected more often and more severely when they do occur. 

Similarly, the number of violent crimes committed in schools has decreased, but the number of 
homicide events involving multiple offenders and victims has increased (Annual Report on School 
Safety, 1998; Fox & Zawitz, 2000). Homicides committed by younger offenders have been more likely 
to involve multiple victims than those committed by older offenders. During the 1992-1993 school 
year there were two incidents, while during the 1997-1998 school year there were six incidents of 
violent crime involving multiple offenders and victims. The number of victims killed in these events 
increased 400% from 4 victims nationally during the 1992-1993 school year to 16 victims during the 
1997-1998 school year. The results are less clear with respect to perceptions of fear at school. The rate 
of students who reported fear while traveling to and from school rose from 4% in 1989 to 7% in 1995 
(Annual Report on School Safety, 1998), and then declined from 7% to 4% between 1995 and 1999 
(Kaufman, et al., 2000). During that same time, the percentage of students ages 12-18 who feared 
being attacked at school decreased from 9% to 5%. However, fewer students reported feeling not at all 
worried about being physically attacked in school. In 1993, 48% reported feeling not at all worried, 
while in 1998, 37% reported feeling not at all worried (Metropolitan Life Insurance Company, 1999). 
Overall, the conflicting message is that fewer people are worried about being personally attacked, but 
fewer people feel safe at school. Gaining an accurate report of rates of violent crimes occurring in 
schools is difficult because some states do not use consistent definitions of violence to collect incident- 
based data (Florida Department of Education, 1999), This results in significant gaps in the 
information about crime rates and trends. Florida is one of three states, however, that uses consistent 
definitions for reporting violent crimes in state, district, and school educational systems. 
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Data collected from Pinellas County, Florida indicated a decrease in the number of violent acts 
against persons (e.g., homicide, sexual battery, robbery, battery, kidnapping) between 1995-1998, 
commensurate with national trends (Florida Department of Education, 1999). Weapons possession, 
however, increased 26% between the 1996-1997 and 1997-1998 school years. The increase in 
incidents of student weapons possession suggests that there may be an increased perception of fear 
among students and staff in Pinellas County, Florida. In order to further investigate the perceptions of 
school safety a survey was administered to guidance counselors in Pinellas County Schools (PCS). 

The Perceptions of School Safety Survey was conducted as part of the Safe Schools Healthy 
Students Initiative (SS/HIS) . The evaluation team in collaboration with the violence prevention 
specialists from PCS designed the instrument to serve as a needs assessment tool. By obtaining the 
staff’s perceptions of safety in their schools, the survey would ultimately help to enhance existing 
strategies and programs dealing with violence and safety issues in the schools. 

Method 

In developing the survey, attempts were made to include as many items as possible that reflect 
issues of concern to the schools. As a result, drawing from the school’s disciplinary referrals data, items 
such as bullying, fighting, verbal threats, and physical violence were included. The survey consisted of 
three sections dealing with: 1) the seriousness of violence in the schools, 2) the effectiveness of current 
programs and strategies, and 3) the staff’s familiarity with SS/HSI programs and strategies, This paper 
focuses on the first section of the survey dealing with the seriousness of violence. Respondents were 
asked to rate 19 items on a five-point Likert scale ranging from “extreme problem” to “not a problem,” 
In addition, they were asked to rate the overall safety of their school, from “very safe” to “very 
dangerous.” The surveys were distributed to the counseling staff of elementary, middle and high 
school in Pinellas County with 10 1 completed surveys returned. The 101 questionnaires represented 
66 elementary, 20 middle, and 14 high schools. In addition, a pilot sample of 100 questionnaires was 
received from teachers and students at one middle school. Only surveys completed by the counseling 
staff are included in the current analysis. 

Preliminary Results 

The results of the overall safety revealed that 60.4% of counselors perceived their school to be “safe” 
and 21% perceived it to be very safe. Results on the first 19 items dealing with the seriousness of violence 
revealed that counselors in general have a positive perception of their schools, For instance, gangs and 
activities in school, as illustrated in Figure 1, were not perceived as problems, Two items were of most 
concern to counselors. These included teasing among students (M = 3,4 reflecting moderate to serious 
problem and bullying among students {M - 2.9) reflecting a moderate problem, To further analyze the 
sample a factor analysis was performed on the initial 19 items. 

Factor Analysis 

The factor analysis was performed using the Maximum Likelihood method with a Varimax 
Rotation, and produced four factors or groups (see Table 1). Factor one, labeled “Child behaviors” 
included ten items that were considered common student behavior problems, Representative items 
include verbal threats among staff with a factor loading of ,86 and physical violence among students 
with a factor loading of .74. Factor two, labeled “Crime” included four items that could be described as 
illegal activities such as students using drugs in school with a factor loading ,95 and gang activity with a 
loading of .88, Factor three, labeled “School administration” consists of two items related to school 
administration that includes lack of administrative support and ineffective discipline policies, Although 
this factor has only two items, they have a high internal reliability (.84), Consequently, further 
development will be done in to this domain. Factor four, labeled “School locale” includes three items 
concerning activities in the neighborhood where school is located including vandalism loading ,72 and 
personal property stolen or destroyed loading .66. 
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Figure 1 

Counselor's Perception of School Safety 



Verbal threats among students 
Verbal threats directed to staff 
Physical violence among students 
Physical violence toward staff 
Students using drugs or alcohol in school 
Drugs being sold in school 
Teasing among students 
Bullying among students 
Gang activity in school 
Personal property stolen 
Vandalism 
Discrimination 
Violence in the community 
Lack of administrative support 
Lack of parental support 
Ineffective discipline policies 
Teachers' ineffectiveness 
Students threatened on bus 
Illegal activity in school 

1.0 2.0 3.0 4.0 5.0 

Not a Problem Minimal Moderate Serious Extreme 




Table 1 

Factor Loading for Four-Factor Solution 



Factor 1. Child Behaviors 


Loadings 


Verbal Threats 


.86 


Verbal threats directed at staff 


.73 


Physical violence among students 


.74 


Physical violence toward staff 


.60 


Teasing among students 


.67 


Bullying among students 


.68 


Lack of parental support in addressing discipline 


.48 


Students threatened on bus and at bus stop 


.52 


Teachers ineffectiveness in addressing discipline problems 


.58 


Discrimination 


.35 


Internal reliability = .89 




Factor 2. Crime 


Loadings 


Students using drugs or alcohol in school 


.95 


Drugs being sold in school 


.88 


Gang activity in school 


.43 


Illegal activity in school 


.70 


Internal reliability = .85 




Factor 3. School Administration 


Loadings 


Lack of administrative support 


.62 


Ineffective discipline policies 


.86 


Internal reliability = .84 




Factor 4. School Locale 


Loadings 


Vandalism 


.72 


Violence in community where school is located 


.44 


Personal property stolen or destroyed at school 


.66 


Internal reliability = .74 
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Discussion 

This preliminary analysis suggests that in addressing the issues of school safety there seems to be at 
least four domains related to the perceptions of school safety. These include troublesome behaviors, 
crime, school administration, and school locale. These factors represent common areas of concern 
regarding violence and safety in the schools. These preliminary data encourage us to expand the 
instrument by including other possible factors and administering it to additional school personnel and 
students. Future plans include the addition of new domains as well as adding new items to some of the 
existing domains. 
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